1. Field of the Invention
The present invention relates generally to digital image processing, and more specifically to a method and system for detecting a moving object in a monitored wide background image by using a single video camera and tracking the object by controlling the panning, tilting and zooming functions of the video camera or PTZ camera.
2. Description of the Related Art
Video monitoring systems are used as a means for protecting an area or an unattended site from unauthorized intrusion. In these systems, a moving object is first detected in a monitored background image and then the target object is magnified and followed. The video image of the target moving object is recorded on a video cassette recorder for later analysis and identification. One example of the prior art video monitoring systems is described in Japanese Patent Publication 11-69342. In this system two video cameras are used, the first camera for detecting a motion in a monitored background image using a subtraction method and the second, PTZ camera for zooming in on and following the detected moving object using the position and size information of the motion obtained by the first camera. However, the use of two video cameras is costly and undesirable.
In Japanese Patent Publication 10-13821, a single video camera with a pan-and-tilt function and a subtraction method are used for initially detecting a motion in a monitored background image. Template matching then proceeds. In this process, the target image is stored in memory as a template and the camera is controlled so that difference between the current image and the template is reduced to a minimum. However, no zooming function is provided in this prior art.
It is therefore an object of the present invention to provide a method and system for following a detected fast moving object by making pan, tilt and zooming operations of a video camera.
A further object of the present invention is to provide a method and a system for using a single video camera for detecting a moving object and following the detected object by making pan, tilt and zooming operations of the single video camera.
Precision tracking is achieved by performing a motion compensation on a preceding frame to compensate for a movement of a background image of a detected moving object due to camera""s tracking motion and estimating the position of the moving object based on the image difference between the motion-compensated frame and a succeeding frame.
According to a first aspect, the present invention provides a method of following a detected moving object using a video camera having panning, tilting and zooming functions, comprising the steps of (a) storing succeeding and preceding frames produced by the camera into first and second memories respectively and controlling the panning, tilting and zooming functions of the video camera according to a position and a size of the moving object, (b) performing a motion compensation on the preceding frame to compensate for a motion of a background image of the moving object so that coordinates of the preceding frame are transformed to coordinates of the succeeding frame, (c) detecting an image difference between the motion-compensated preceding frame and the succeeding frame and determining from the image difference a position and a size of the moving object, and (d) repeating the steps (a) to (c) by using the position and the size determined by the step (c) when the step (b) is repeated.
According to a second aspect, the present invention provides a method of following a detected moving object using a video camera having panning, tilting and zooming functions. The method comprises the steps of (a) producing a template representing a position and a size of the detected moving object, (b) storing succeeding and preceding frames produced by the camera into first and second memories respectively and controlling the panning, tilting and zooming functions of the video camera according to the stored position and size of the moving object, (c) selecting one of first, second and third tracking modes depending on a movement of the detected moving object. If the first tracking mode is selected, a motion compensation is performed on the preceding frame to compensate for a motion of a background image of the moving object so that coordinates of the preceding frame are transformed to coordinates of the succeeding frame, and an image difference between the motion-compensated frame and the succeeding frame is detected and a position and a size of the moving object are determined from the image difference, and the step (a) is repeated. If the second tracking mode is selected, the size of the template is transformed according to the zooming operation of the video camera caused by the step (b) as an updated size of the moving object, and a position of the moving object is detected that matches the transformed template, and the step (b) is repeated. If the third tracking mode is selected, the size of the template is transformed according to the zooming operation of the video camera caused by the step (b) as an updated size of the moving object, a motion compensation is performed on the preceding frame to compensate for a motion of a background image of the moving object so that coordinates of the preceding frame are transformed to coordinates of the succeeding frame, an image difference is detected between the motion-compensated frame and the succeeding frame, a position of the moving object is determined by using the transformed template and the image difference and the step (a) is repeated.
According to a third aspect of the present invention, there is provided an image tracking system comprising a video camera having panning, tilting and zooming functions, first and second memories for storing succeeding and preceding frames produced by the camera, respectively, tracking control circuitry connected to the video camera for detecting a moving object and performing a motion compensation on the preceding frame to compensate for a motion of a background image of a moving object so that coordinates of the preceding frame are transformed to coordinates of the succeeding frame, detecting an image difference between the motion-compensated frame and the succeeding frame, and determining from the image difference a position and a size of the moving object, and camera control circuitry for controlling the panning, tilting and zooming functions of the video camera according to the position and the size of the moving object.
According to a fourth aspect, the present invention provides an image tracking system for tracking a detected moving object, comprising:
a video camera having panning, tilting and zooming functions;
a template memory;
first and second memories for storing succeeding and preceding frames produced by the camera, respectively; and
tracking control circuitry for performing the functions of:
a) storing a position and a size of the detected moving object as a template in the template memory;
b) controlling the panning, tilting and zooming functions of the video camera according to the stored position and size of the moving object;
c) selecting one of first, second and third tracking modes depending on a movement of the detected moving object;
d) if the first tracking mode is selected, performing a motion compensation on the preceding frame to compensate for a motion of a background image of the moving object so that coordinates of the preceding frame are transformed to coordinates of the succeeding frame;
e) detecting an image difference between the motion-compensated frame and the succeeding frame and determining from the image difference a position and a size of the moving object, and returning to the function (a);
f) if the second tracking mode is selected, transforming the size of the template according to the zooming operation of the video camera caused by the function (b) as an updated size of the moving object, and determining a position of the moving object that matches the transformed template, and returning to the function (b);
g) if the third tracking mode is selected, transforming the size of the template according to the zooming operation of the video camera caused by the function (b) as an updated size of the moving object;
h) performing a motion compensation on the preceding frame to compensate for a motion of a background image of the moving object so that coordinates of the preceding frame are transformed to coordinates of the succeeding frame; and
i) detecting an image difference between the motion-compensated frame and the succeeding frame, determining a position of the moving object by using the transformed template and the image difference and returning to the function (a).
Preferably, the interval between the first and second frames is varied as a function of the size or speed of movement of the moving object.